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Abstract 

The LMC complexity, an indicator of complexity based on a probabilistic de- 
scription, is revisited. A straightforward approach allows us to establish the time 
evolution of this indicator in a near-equilibrium situation and gives us a new insight 
for interpreting the LMC complexity for a general non equilibrium system. Its re- 
lationship with the Renyi entropies is also explained. One of the advantages of this 
indicator is that its calculation does not require a considerable computational effort 
in many cases of physical and biological interest. 
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1 Shannon information 



Entropy plays a crucial theoretical role in physics of macroscopic equilibrium sys- 
tems. The probability distribution of accessible states of a constrained system in 
equilibrium can be found by the inference principle of maximum entropy [1]. The 

macroscopic magnitudes and the laws that relate them can be calculated with this 
probability distribution by standard statistical mechanics techniques. 

The same scheme could be thought for extended systems far from equilibrium, 
but in this case we do not have neither a method to find the probability distribution 
nor the knowledge of the relevant magnitudes bringing the information that can 
predict the system's behavior. It is not the case, for instance, with the metric 
properties of low dimensional chaotic systems by means of the Lyapunov exponents, 
invariant measures and fractal dimensions [2] . 

Shannon information or entropy H [3] can still be used as a magnitude in a 
general situation with N accessible states: 



with K a positive real constant and pi the normalized associated probabilities, 
Y^f=iPi — 1. An isolated system in equilibrium presents equiprobability, Pi — 1/N 
for all i, among its accessible states and this is the situation of maximal entropy. 



If the system is out of equilibrium, the entropy H can be expanded around this 
maximum Hmax'- 



N 



H = -KY^Pdogpi 



(1) 



Hmax ^ K log N. 
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H{pi,P2, - ■ ■ ,Pn) = KlogN TT^yP- 



i=l 
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where the quantity D = J2i{Pi — ^/^Y, that we call disequilibrium, is a kind of 
distance from the actual system configuration to the equilibrium. If the expression 
(3) is multiplied by H we obtain: 

H^^H- Hma. -^H-D + K^f{N,pi), (4) 

where f{N,pi) is the entropy multiplied by the rest of the Taylor expansion terms, 
which present the form Y,i{Npi — 1)"^ with m > 2. If we rename C — H ■ D, 

C^cte-H-{H^^-H) + Kf{N, p,) , (5) 

with cte~^ — NK/2 and / = 'if /N. The idea of distance for the disequilibrium is 
now clearer if we see that D is just the real distance D ~ {Hmax — H) for systems in 
the vicinity of the equiprobability. In an ideal gas we have H ~ Hmax and ~ 0, 
then C ~ 0. Contrarily, in a crystal if ~ and D ~ 1, but also C ~ 0. These two 
systems are considered as classical examples of simple models and are extrema in a 
scale of disorder [H) or disequilibrium {D) but those should present null complexity 
in a hypothetic measure of complexity. This last asymptotic behavior is verified by 
the variable C (Fig. 1) and C has been proposed as a such magnitude [4]. We 
formalize this simple idea recalling the recent definition of LMC complexity in the 
next section. 

Let us see another important property arising from relation (5). If we take 
the time derivative of C in a neighborhood of equilibrium by approaching C ~ 
H{Hmax ~ H), then we have 

dC dH 

- ~ -27/_-. (6) 

The irreversibility property of H implies that ^ > 0, the equality occurring only 
for the equipartition, therefore 



Hence, in the vicinity of Hj^ax, LMC complexity is always decreasing on the evolution 
path towards equilibrium, independently of the kind of transition and of the system 
under study. This does not forbid that complexity can increase when the system is 
very far from equilibrium. In fact this is the case in a general situation as it can be 
seen, for instance, in the system presented in Ref [5]. 



2 LMC complexity 

Let us assume that at the scale of observation a system has N accessible states 
{xi,X2, ...,xn} (N-system) and a probability distribution {pi,p2, ■■■,Pn} oi each state 
{pi 7^ for all i) . Then, at this level of description the knowledge of the underlying 
physical laws is "expressed" by a probability distribution among the accesible states. 
Shannon [3] demonstrated that the only function that gives the information of a 
system under the most elementary assumptions is H = —K Y.f=iPi ^og pi. It is easy 
to find out that the information H contained in a crystal is ~ 0, while for an 
isolated gas pi ~ 1/N and then Hg ~ KlogN that represents the maximum of 
information for a system with N states. Any other N-system will have information 
between those two extrema. 

The disequilibrium D oi a system can be taken as some kind of distance to an 
equiprobable distribution. Two conditions are required to this magnitude D > 
(positive measure of complexity) and D — in the limit of equiprobability. The 
easier solution is to add the quadratic distances of each state to the equiprobability, 
i.e., D — Y^i^iiPi ~ jfT- This function will be a maximum for a crystal and zero (by 
construction) for an ideal gas. Any other N-system will have disequilibrium between 
these two extrema. 



Following the discussion in the introduction the definition of LMC complexity 
(C) is presented [4]: 

C^H-D^- (KYJ>^^ogp)| ■ (jjp, - 1)2^ (8) 

This definition fits the intuitive arguments and gives C ~ for the two systems (a 
perfect crystal and the ideal gas) discussed herein. Any other system will have an 
intermediate behavior and therefore C > 0. At different scales a different number of 

states (a different probability distribution) are accesiblc to the system and therefore 
different H and D. Therefore, the magnitude complexity defined is scale- dependent 
as expected. 

Direct simulations of the definition of C and its comparison with H for several 
systems in different contexts has been presented in Ref. [4, 5]. For a 2-system an 
analytical expression for the curve C{H) is obtained. For N > 2 the relationship be- 
tween H and C is not univoque anymore. Many different distributions {pi} can have 
associated the same information H but different complexity C. But a similar form 
as the one found in the 2-system case is recovered for a N-system when the maxi- 
mum complexity Cmax{H) is calculated for each H. These curves, represented in a 
normalized way CmaxiH), with normalization K — l/logA?^, have been numerically 
computed for the cases = 2, 3, 5, 7 in Ref. [4]. When A^ tends towards infinity [5] 
the maximum disequilibrium scales as {1 — H)^ and the maximum complexity tends 
to 

lim Cmax^H-{l-H)\ (9) 

AT— >oo 

The limit of the minimum disequilibrium and complexity vanishes, 

lim Cmin = 0. (10) 
AT— »oo 

In general, in the limit A^ — > oo, the complexity is not a trivial function of the 
entropy, in the sense that for a given H there exists a range of complexities between 



and Caiax{H)) (Eqs. (9-10)). In particular, in this asymptotic limit, the maximum 
of Cmax is found when H — 1/3, which gives a maximum of the maximum complexity 
of Cmax = 4/27 and confirms the numerical calculation of Ref. [6]. This value is 
reached when the distribution presents a dominant state with probability Pmax = | 
and the rest of the infinitely many states is a uniform 'sea' of equal probability. We 
say that this distribution is a type-hke 'Rey-Pueblo' configuration. 

An attempt of extending the LMC complexity for continuous systems has been 
performed in Ref. [7]. When the number of states available for a system is a contin- 
uum then the natural representation is a continuous distribution. In this case, the 
entropy can become negative. The positivity of C for every distribution is recovered 
by taking the exponential of H. If we define C = H ■ D = ■ D as an extension of C 
to the continuous case interesting properties characterizing the indicator C appear. 
Namely, its invariance under translations, rescaling transformations and replication 
convert (7 in a good candidate to be considered as an indicator bringing essential 
information about the statistical properties of a continuous system. 

The most important point is that the definition should work in systems out of 
equilibrium. We present a significative example [4] for the logistic map, the typical 
chaotic system in which the transition from chaos to a 3-period orbit via intermit- 
tency is know to be extremely "complex". This is due to the fact that the intermit- 
tent bursts are more and more improbable and impredictible when the transition 
point is approached. After the transition point a period three orbit stabilizes and 
the dynamics becomes simple. (Complexity is calculated by means of the binary 
sequences issued from the numerical simulations of the map, using the natural par- 
tition). We see in Fig. 2 that the values of C in the intermittency transition point 
recalls a second order phase transition. 



3 Renyi entropies 



Generalized entropies have been introduced by Renyi [8] in the form of 

^. = Y^iog(Epf)> (11) 

where q is an index running over all the integer values. By differentiating Iq with 
respect to g a negative quantity is obtained independently of then Iq monotonously 
decreases when q increases. 

The Renyi entropies are an extension of the Shannon information H. In fact, H 
is obtained in the limit g — > 1: 

N 



H ^ h^limlq^ - Vpi logpi, (12) 

1-^^ i=i 

where the constant K of Eq. (1) is considered to be the unity. The disequilibrium 
D is also related with I2 — — log (Z^^i pj^ ■ We have that 



1 . 1 



^ = Ep? - 77 = - (13) 



N N 

then the LMC complexity is 



C^H.D^h.[e-^-^\-]. (14) 



N 

The behavior of C in the neighborhood of H^^x takes the form 

C~l(log2iV-/i/2), (15) 

The obvious generalization of the Renyi entropies for a normalized continuous dis- 
tribution p{x) is 

Iq^J^l0gj\p{x)Ydx. (16) 

Hence, 

H = I^^ - j p{x) logp{x)dx, (17) 
D = = J [p{x)]^dx. (18) 



The dependence oi C = ■ D with /i and I2 yields 

logC' = (/i-/2). (19) 

This indicates that a family of different indicators could derive from the differences 
established among Renyi entropies with different g-indices. 

The invariance of C under rescaling transformations implies that this magnitude 
is conserved in many different processes. For instance, the initial Gaussian-like 
distribution will continue to be Gaussian in a classical diffusion process. Then C is 
constant in time: ^ = 0, and we have: 

f-f- 

The equal losing rate of Ii and I2 is the cost to be paid in order to maintain the shape 
of the distribution associated to the system and, hence, all its statistical properties 
will remain unchanged during its time evolution. 



4 Discussion and conclusions 



A definition of complexity (LMC complexity) based on a probabilistic description of 
physical systems has been revisited. This definition contains basically an interplay 
between the information contained in the system and the distance to equipartition 
of the probability distribution representing the system. Besides giving the main 
features of an intuitive notion of complexity, we showed that it allows to successfully 
discern situations considered as complex in systems of a very general interest. In 
particular it proved to be useful for quantifying the complexity in some statistical 
and laser systems [9, 10]. Its relationship with the Shannon information and the 
generalized Renyi entropies has been shown to be explicit. Moreover it has been 



possible to establish the decrease of this magnitude when a general system evolves 
from a near-equilibrium situation to the equipartition. 

Many different notions of complexity have been proposed until now, mainly in 
the context of computational and social sciences. Most of these definitions present 
either operational difficulties or arise logical problems. The main advantage of LMC 
complexity is its generality and the fact that it is operationally simple and do not 
require a big amount of calculations. This advantage has been worked out in different 
examples, such as the study of the time evolution of C for a simplified model of an 
isolated gas, the "tetrahedral gas" [5], the slight modification of C as an effective 
method by which the complexity in hydrological systems can be identified [11], 
the attempt of generalize C in a family of simple complexity measures [12], some 
statistical features of the behavior of C for DNA sequences [13] and some wavelet- 
based informational tools used to analyze the brain electrical activity in epilectic 
episodes in the plane of coordinates {H, C) [14] . We are convinced that it provides 
a useful way of thinking and it can help in the future to gain more insight on the 
physical grounds of models with potential biological interest. 
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Figure Captions 

1. Sketch of physical intuition of the magnitudes "information" (H) and "dis- 
equilibrium" (D) between the two simple systems: crystal and ideal gas. Also, 
intuitive behavior required for the magnitude complexity. The quantity C — H ■ D 
was proposed as such a magnitude. 

2. Behavior of the complexity C in the transition point {pt ~ 3.8284) where the 
system (logistic map) goes from a chaotic dynamics by intermittency (p < pt) to a 
period three orbit (p > pt) . 



